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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 

Listing of Claims: 

Please amend claims 1-3, 10-12, 18-20, and 29, as follows: 

1 . (Currently Amended) A method of recovering depth information for pixels of a base image 
representing a view of a scene, the method comprising the steps of: 

detecting a plurality of pixels in a base image that represents a first view of a scene,, 
wherein the first view is one of at least three views of a scene ; 

determining 3-D depth of the plurality of pixels in the base image by matching 
correspondence to a plurality of pixels in a plurality of images , each image representing a 
plurality of one of the at least three views of the scene , wherein each of the at least three views of 
the scene are situated in a non-linear arrangement and are further oriented in a plurality of non- 
parallel planes relative to each other ; and 

tracing pixels in a virtual piecewise continuous depth surface by spatial propagation 
starting from the detected pixels in the base image by using the matching and corresponding 
plurality of pixels in the plurality of images to create the virtual piecewise continuous depth 
surface viewed from the base image, each successfully traced pixel being associated with a depth 
in the scene viewed from the base image. 
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2. (Currently Amended) The method of Claim 1, wherein the step of determining the 3-D 
depth comprises the steps of: 

detecting a plurality of image pixels in a first image corresponding to a first view of a 

scene; 

detecting a plurality of image pixels in at least a second image corresponding to a 
respective at least a second view of the scene, wherein the at least a second image deviates from 
the first image as a result of camera relative motion; 

determining a first two-view correspondence between the plurality of detected image 
pixels in the first image and a plurality of detected image pixels in one of the at least a second 
image resulting in a fet potential two-view match set of candidate image pixels between the 
first image and the one of the at least a second image; and 

concatenating a plurality of two-view match sets to form a first multiple-view potential 
match set; and 

determining a second multiple-view potential match set that is a refinement of the first 
multiple-view potential match set. 

d e t e rmining a multipl e vi e w correspond e nc e b e tw ee n the plurality of d e t e ct e d imag e 
pixels in the first imag e and the plurality of d e t e ct e d imag e pixels in th e at l e ast a second imag e , 
the multiple - vi e w corr e spondence b e ing a r e fin e m e nt of th e first two - vi e w corr e spond e nc e , 
resulting in a s e cond potential match s e t of candidat e image pix e ls b e tw ee n the first imag e and 
the at least a second imag e , wherein the second potential match set is based at least in part on a 
computation of r e proj e ction e rror for matched pixels that r e sult e d from a proj e ctiv e 
reconstruction of th e s e cond potential match s e t. 

3. (Currently Amended) The method of claim 2, wherein the second multiple-view potential 
match set is based at least in part on a least median of squares computation of the reprojection 
errors related to matched pixels in the s e cond first multiple-view potential match set. 

4. (Original) The method of Claim 1 , wherein the tracing step comprises the step of: 

propagating a front of a virtual piece of a continuous depth surface to at least one 
neighboring pixel starting from the detected pixels in the base image. 
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5. (Original) The method of Claim 4, wherein the tracing step comprises the step of: 

determining when a boundary is reached between two propagating fronts of virtual pieces 
of a continuous depth surface. 

6. (Original) The method of Claim 4, wherein the tracing step comprises the steps of: 

comparing the matching costs of the two propagating fronts about the reached boundary; 

and 

stopping the propagation of the front with the higher compared matching cost. 

7. (Original) The method of Claim 1, wherein the tracing step comprises the step of: 

propagating a front of a virtual piece of a continuous depth surface to at least one 
neighboring pixel surrounded by a predefined size window in the continuous depth surface. 

8. (Original) The method of Claim 7, wherein the tracing step comprises the step of: 

determining when a boundary is reached between two propagating fronts of virtual pieces 
of a continuous depth surface. 

9. (Original) The method of Claim 7, wherein the tracing step comprises the steps of: 

comparing the matching costs of the two propagating fronts about the reached boundary; 

and 

stopping the propagation of the front with the higher compared matching cost. 

10. (Currently Amended) The method of Claim 9, wherein the matching cost is determined 
by computing the summation of all the normalized cross-correlations between a first image 
window of a pre-determined size in the base view, and a second image window of the same pre- 
determined size in a one of at least one reference view. 
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1 1 . (Currently Amended) The method of Claim 10, further comprising a step of: 

rectification of at least one pair of images corresponding to the base view of the scene 
and one of the at least one of th e reference views. 

12. (Currently Amended) An image processing system comprising: 

a memory; 

a controller/processor electrically coupled to the memory; 

an image matching module, electrically coupled to the controller/processor and to the 
memory, for providing a plurality of seed pixels that represent 3-D depth of the plurality of 
pixels in the base image view of a scene by matching correspondence to a plurality of pixels in a 
plurality of images representing a plurality of views of the scene , wherein the base image view is 
one of at least three views of the scene, each of the at least three views of the scene being 
situated in a non-linear arrangement and are further oriented in a plurality of non-parallel planes 
relative to each other : and 

a propagation module, electrically coupled to the controller/processor and to the memory, 
for tracing pixels in a virtual piecewise continuous depth surface by spatial propagation starting 
from the provided plurality of seed pixels in the base image by using the matching and 
corresponding plurality of pixels in the plurality of images to create the virtual piecewise 
continuous depth surface viewed from the base image, each successfully traced pixel being 
associated with a depth in the scene viewed from the base image. 

13. (Original) The image processing system of claim 12, further comprising at least one camera 
interface, electrically coupled to the controller/processor, for sending image information from at 
least one camera to the controller/processor. 

14. (Original) The image processing system of claim 12, wherein the controller/processor, the 
memory, the image matching module, and the propagation module, are implemented in at least 
one of an integrated circuit, a circuit supporting substrate, and a scanner. 
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15. (Original) The image processing system of Claim 12, wherein the propagation module is 
further for determining when a boundary is reached between two propagating fronts of virtual 
pieces of a continuous depth surface. 

16. (Original) The image processing system of Claim 15, wherein the propagation module is 
further for comparing the matching costs of the two propagating fronts about the reached 
boundary; and 

stopping the propagation of the front with the higher compared matching cost. 

17. (Original) The image processing system of Claim 12, wherein the propagation module is 
further for propagating a front of a virtual piece of a continuous depth surface to at least one 
neighboring pixel surrounded by a predefined size window in the continuous depth surface. 

18. (Currently Amended) A computer readable medium including computer instructions for a 
3-D image reconstruction computer system, the computer instructions comprising instructions 

♦ for: 

detecting a plurality of pixels in a base image that represents a first view of a scene,, 
wherein the first view is one of at least three views of a scene ; 

determining 3-D depth of the plurality of pixels in the base image by matching 
correspondence to a plurality of pixels in a plurality of images , each image representing a 
plurality of one of the at least three views of the scene , wherein each of the at least three views of 
the scene are situated in a non-linear arrangement and are further oriented in a plurality of non- 
parallel planes relative to each other ; and 

tracing pixels in a virtual piecewise continuous depth surface by spatial propagation 
starting from the detected pixels in the base image by using the matching and corresponding 
plurality of pixels in the plurality of images to create the virtual piecewise continuous depth 
surface viewed from the base image, each successfully traced pixel being associated with a depth 
in the scene viewed from the base image. 
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19. (Currently Amended) The computer readable medium of claim 18, wherein the step of 
determining the 3-D depth comprises the steps of: 

detecting a plurality of image pixels in a first image corresponding to a first view of a 

scene; 

detecting a plurality of image pixels in at least a second image corresponding to a 
respective at least a second view of the scene, wherein the at least a second image deviates from 
the first image as a result of camera relative motion; 

determining a first two-view correspondence between the plurality of detected image 
pixels in the first image and a plurality of detected image pixels in one of the at least a second 
image resulting in a ffet potential two-view match set of candidate image pixels between the 
first image and the one of the at least a second image; and 

concatenating a plurality of two-view match sets to form a first multiple-view potential 
match set; and 

determining a second multiple-view potential match set that is a refinement of the first 
multiple-view potential match set. 

d e t e rmining a multipl e- vi e w corr e spond e nc e b e tw ee n the plurality of detected imag e 
pixels in th e first imag e and the plurality of d e t e ct e d imag e pix e ls in th e at l e ast a s e cond image, 
the multipl e- vi e w corr e spondence being a r e fin e m e nt of th e first two - vi e w correspond e nce, 
resulting in a s e cond pot e ntial match s e t of candidat e imag e pix e ls b e tw ee n the first imag e and 
the at least a second image, wherein the second potential match set is based at least in part on a 
computation of r e proj e ction error for match e d pix e ls that r e sult e d from a proj e ctiv e 
reconstruction of th e second potential match s e t. 

20. (Currently Amended) The computer readable medium of claim 19, wherein the second 
multiple-view potential match set is based at least in part on a least median of squares 
computation of the reprojection errors related to matched pixels in the s e cond first multiple-view 
potential match set. 
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21. (Original) The computer readable medium of claim 18, wherein the tracing step comprises 
the step of: 

propagating a front of a virtual piece of a continuous depth surface to at least one 
neighboring pixel starting from the detected pixels in the base image. 

22. (Original) The computer readable medium of Claim 21, wherein the tracing step comprises 
the step of: 

determining when a boundary is reached between two propagating fronts of virtual pieces 
of a continuous depth surface. 

23. (Original) The computer readable medium of Claim 21, wherein the tracing step comprises 
the steps of: 

comparing the matching costs of the two propagating fronts about the reached boundary; 

and 

stopping the propagation of the front with the higher compared matching cost. 

24. (Original) The computer readable medium of Claim 18, wherein the tracing step comprises 
the step of: 

propagating a front of a virtual piece of a continuous depth surface to at least one 
neighboring pixel surrounded by a predefined size window in the continuous depth surface. 

25. (Original) The computer readable medium of Claim 24, wherein the tracing step comprises 
the step of: 

determining when a boundary is reached between two propagating fronts of virtual pieces 
of a continuous depth surface. 
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26. (Original) The computer readable medium of Claim 24, wherein the tracing step comprises 
the steps of: 

comparing the matching costs of the two propagating fronts about the reached boundary; 

and 

stopping the propagation of the front with the higher compared matching cost. 

27. (Original) The computer readable medium of Claim 26, wherein the matching cost is 
determined by computing a normalized cross-correlation between the windows associated with 
the two propagating fronts about the reached boundary. 

28. (Original) The computer readable medium of Claim 27, further comprising a step of: 

rectification of at least one pair of images corresponding to at least one pair of views of 
the scene. 
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29. (Currently Amended) A method of recovering depth information for pixels of a base 
image representing a view of a scene, the method comprising the steps of: 

tracing at least one parameter surface, each of the at least one parameter surface traced 
starting from at least one predetermined seed pixel point; and 

calculating a derivative of function E(g) with respect to parameter g by using finite 
difference to minimize the following equation 



is a reference image, 
To is a base image. 

(w, v) is a pixel point closest to Pj(Co+gL(iip v y )); 

Co+gL(Uj 9 vj) is a 3-D point that projects to (w,, vj) in the base view, and to Pi(Co+gL(uj, vj)) in the 
z th reference view; 

Co is the base image camera's center of projection; 

L{uj,Vj) is the unit vector from Co to the point on the image plane that corresponds the pixel 
(uj,Vj); and 

g is a depth parameter. 




where NCCl/^w^ji/o^^v^jJis a normalized cross-correlation between /,(w,v) and 7 0 (w /? v y ); 

v) is a first window of size coxa) centered at pixel (w,v) in I h 
I 0 (Uj,Vj) is a second window of size wxw coxa) centered at pixel (uj,vj) in Io, 
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